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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.1 14, and the fee set 
forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 2/14/08 
has been entered. 

Response to Amendment 

1. This office action is in response to an amendment filed on 2/14/2008. 

2. Claims 1, 17, 40 and 41 has been amended by the applicant. 

3. Claims 2-14, 22 and 24-39 are original. 

4. Claims 15, 16, 18-21, 23 and 42 have been cancelled. 

Allowable Subject Matter 

In regards to claim 30, Seki and Fels do not teach that the image conversion unit 
partially changes a rate of the new moving-picture frame to be outputted from said image 
data output unit in a manner such that, according to attribute values of image regions that 
constitute the two-dimensional images, the cut surface is varied in time with different 
speed for each of the image regions, as recited in claim 13, and also do not teach that the 
attribute value is a value that indicates the order of approximation relative to a desired 
image pattern, as recited in claims 29 and 30. Therefore claims 13, 29 and 30 are 
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objected to as being dependent upon a rejected base claim, but would be allowable if 

rewritten in independent form including all of the limitations of the base claim and any 

intervening claims. 

Claims 1-14, 27, 29, 31, 33, 35 and 40 are allowed. The following is an 

examiner's statement of reasons for allowance: 

The prior art, Seki (JP 09-035040), does not teach the limitations of claims 1 and 
40. In regards to claims 1 and 40 4, Seki teaches original moving pictures as two- 
dimensional images that vary along time axis, and when the moving pictures are 
expressed, in a virtual manner, as a box space formed by the two-dimensional images 
and the time axis, cutting the box space by a surface that contains a plurality of points 
each of which differs from the other in time value (H001 1 lines 5-9: ". . .there is a time 
axis (T-axis) perpendicular to both X-axis and Y-axis, by setting the images along this 
axis, it is possible to construct the three-dimensional image shown in Fig. 3, that is, 
time-space image I(x, y; t). In step SP2, said time-space image I(x, y; t) is cut by a plane 
parallel to the time axis, and the image appearing on the cut plane is taken as 
cross-sectional image...", Fig. 3) and varying the cut surface in time (TJ0010 lines 4-5: 
". . .the cross-section of the time-space image taken by a plane parallel to the time 
axis"). However, none of the prior art teaches or suggests projecting a first image that 
appears on the cut surface onto a first plane perpendicular to the time axis, projecting a 
second image that appears on the varied cut surface onto a second plane perpendicular to 
the time axis, and outputting the first and second images appearing on the first and 
second planes as new moving pictures, therefore claims 1-3 and 40 are allowable. 
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The prior art, Seki (JP 09-035040), does not teach the limitations of claim 4. In 
regards to claim 4, Seki teaches original moving pictures as two-dimensional images 
that vary along time axis, and when the moving pictures are expressed, in a virtual 
manner, as a box space formed by the two-dimensional images and the time axis, cutting 
the box space by a surface that contains a plurality of points each of which differs from 
the other in time value (Tf00 1 1 lines 5-9: ". . .there is a time axis (T-axis) perpendicular 
to both X-axis and Y-axis, by setting the images along this axis, it is possible to 
construct the three-dimensional image shown in Fig. 3, that is, time-space image I(x, y; 
t). In step SP2, said time-space image I(x, y; t) is cut by a plane parallel to the time axis, 
and the image appearing on the cut plane is taken as cross-sectional image. . .", Fig. 3) 
and varying the cut surface in time (1(0010 lines 4-5: ". . .the cross-section of the 
time-space image taken by a plane parallel to the time axis"). However, none of the 
prior art teaches or suggests projecting a first image that appears on the cut surface onto 
a first plane perpendicular to the time axis, projecting a second image that appears on 
the varied cut surface onto a second plane perpendicular to the time axis, and outputting 
the first and second images appearing on the first and second planes as new moving 
pictures, therefore claims 4-14, 27, 29, 31, 33 and 35 are allowable. 

Claim Rejections - 35 USC §103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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Claims 17, 22, 24-26, 28, 32, 34, 36 and 41 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Seki (JP 09-035040). 

Regarding claim 17, Seki teaches an image generating method (](0001 lines 2-3: 
". ..invention pertains to an image processing method. . ."), including: 

reading out, for a first in-picture position of a first image contained in a first target 
frame in original moving pictures, first data that correspond to the in-picture position 
(1(0012 lines 1-2 and 6-1 1 : "...a portion of the image of moving object (21) contained in 
the original consecutive images must appear as the trace. . . This is called trace 
cross-sectional image L(s, 9; t), and it is found that this image becomes a place 
containing the velocity vector of the moving object. . . This plane completely contains the 
information pertaining to the movement of the object. . .", where a certain region from all 
the frames is captured, Fig. 4); 

synthesizing the first data in a ratio according to an attribute value of the first 
image (](0012 lines 1-2 and 8-11: "On said cross-sectional image C(d, t; 9), a portion of 
the image of moving object (21) contained in the original consecutive images must 
appear as the trace... this trace cross-sectional image L(s, 9; t) is an image obtained by 
cutting said time-space image I(x, y; t) with a helix plane along the movement direction 
of the object. This plane completely contains the information pertaining to the movement 
of the object. . .", where an attribute value, such as the proportional difference in the 
position of the object over time, is tracked and presented in a synthesized image that 
displays the movement of the object over a time interval, Fig. 5); 
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reading out, for a second in-picture position of a second image contained in a 
second target frame in the original moving pictures, data that correspond to the second in- 
picture position (TjOOl 1 lines 8-1 1 : ". . .the image appearing on the cut plane is taken as 
cross-sectional image C. . .plural cross-sectional images C(d, t; 0) can be obtained. . . ", 
in which several projected cross sectional images containing in-picture positions, Fig. 5, 
may be obtained and appear on the cut surface); 

synthesizing the second read-out data in a ratio according to an attribute value of 
the second image (^[001 1 lines 8-1 1 : ". ..the image appearing on the cut plane is taken as 
cross-sectional image C. . .plural cross-sectional images ...can be obtained. . . " and 
][0012 lines 1-2 and 8-11: "0« said cross-sectional image C(d, t; 9), a portion of the 
image of moving object (21) contained in the original consecutive images must appear as 
the trace. ..this trace cross-sectional image L(s, 6; t) is an image obtained by cutting said 
time-space image I(x, y; t) with a helix plane along the movement direction of the object. 
This plane completely contains the information pertaining to the movement of the 
object. . .", where an attribute value, such as the proportional difference in the position of 
the object over time, is tracked and presented in a synthesized image that displays the 
movement of the object as imaged in second, or subsequent consecutively captured 
images over a time interval, Fig. 5); and 

though Seki does not specifically teach forming new moving images by 
sequentially outputting frames formed in the synthesizing, one skilled in the art would 
have understood that output of several synthesized images would enable a user to animate 
the movement of several objects in a sequence of images through generation of several 
synthesized images for each group that corresponds to an object in sequence of images 
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(1(0015 lines 1-5: "When there are plural moving objects in the cross-sectional images, 
plural groups of the cross-sectional images are prepared corresponding to the respective 
objects, to get the trace cross-sectional image for each object", whereby a special effect 
may then be visualized showing the movement of several objects present in the sequence 
of images). 

Regarding claims 22, 24 and 41, Seki teaches an image generating apparatus 
which includes an image memory, an image conversion unit and an image data output 
unit (1(001 1 lines 1-3: ". ..a camcorder is used to take the consecutive images that are 
input to an image processor ...as shown in Figure 1, as the images. ..are represented. . .", 
in which an image pickup apparatus enables captured images, thereby stored on the 
apparatus, to enable image display conversion processing enabling the images to be 
output), though Seki does not explicitly teach a recording medium, one skilled in the art 
would have understood that the device used to capture the images (1(001 1 lines 1-2: 
". . .the consecutive images that are input to an image processor", Fig. 4, stores a 
program to execute the image generation, Fig. 4, as recited in the preamble of claim 41), 

wherein said image memory (1(001 1 lines 1-2: "As shown in Fig. I, for example, a 
camcorder is used to take the consecutive images that are input to an image processor", 
in which the images are collected by an image pickup device and are thereby stored in 
an image memory, as disclosed in claims 22 and 24), records, in sequence, original 
moving pictures for each frame, wherein said image conversion unit determines, for each 
in-picture position of an image contained in a target frame (1(0012 lines 1-2 and 8-11: 
"On said cross-sectional image C(d, t; 9), a portion of the image of moving object (21) 
contained in the original consecutive images must appear as the trace. ..this trace 



Application/Control Number: 10/693,231 Page 8 

Art Unit: 2628 

cross-sectional image L(s, 9; t) is an image obtained by cutting said time-space image 
I(x, y; t) with a helix plane along the movement direction of the object. This plane 
completely contains the information pertaining to the movement of the object. . .", wherein 
for the successive frames, the position of an object in image is tracked, Fig. 4), a plurality 
of frames at predetermined time intervals from the frames recorded in said image 
memory (^[00 13 lines 1-2: ". . .all of the consecutive images within a prescribed time are 
obtained beforehand", where fames are captured over a predetermined time interval, 
therefore the time intervals between the frames is predetermined, Fig. 2), 

wherein said image conversion unit reads out, from the plurality of frames, data 
that correspond to the in-picture position and synthesizes the data in a ratio according to 
an attribute value fl|0012 lines 1-2 and 6-1 1 : "...a portion of the image of moving object 
(21) contained in the original consecutive images must appear as the trace. . . This is 
called trace cross-sectional image L(s, 6; t), and it is found that this image becomes a 
place containing the velocity vector of the moving object. . . This plane completely 
contains the information pertaining to the movement of the object. . .", where a certain 
region from all the frames is captured, as shown in Fig. 4, where an attribute value, such 
as the proportional difference in the position of the object over time, is tracked and 
presented in a synthesized image that displays the movement of the object over a time 
interval, Fig. 5), and 

wherein said image data output unit sequentially outputs the synthesized and 
reconstructed image data (Fig. 5). 

Regarding claim 25, Seki teaches that the target frame or at least one of frames is 
at least one of a previous frame in time or a subsequent frame in time with respect to a 
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reference frame which should have been naturally outputted by said image data output 
unit from said image memory (]|0006 lines 6-7: ". ..all of the consecutive images over a 
prescribed time are set side-by-side in time to form a three-dimensional time-space 
image. . .", where the frames are successively located along a time axis, therefore a 
particular frame that is presently analyzed would have a frame from the past in reference 
to a current frame). 

Regarding claim 26, Seki teaches that for each in-picture position of the images 
contained in the target frame, the image conversion unit or processor adds a 
predetermined pixel value in accordance with an attribute value thereof (TJ0016 lines 3-7: 
". . .cutting of the initial time-space image is performed in all directions., .the cutting 
plane is made of a helix plane along the movement trace of the object, and this plane 
completely contains the movement vector of the object, and it contains all of the 
information about the movement trace", where a predetermined cut is performed on the 
surface containing a position, or pixel value, within the frames to track the movement of 
the object in accordance with an attribute value, such as the specified time interval of the 
frames, ](001 1 lines 1-5: ". ..a camcorder is used to take the consecutive images. . .as the 
images at an instant (11, 12, 13) shown in Fig. 2 are represented as I(x, y) with the 
orthogonal coordinates of X-axis and Y-axis, all of the images obtained are set 
side-by-side in time sequence.''''). 

Regarding claim 28, Seki teaches an attribute value is a depth value 1 1 lines 
5-7: ". ..there is a time axis (T-axis) perpendicular to both X-axis and Y-axis, by setting 
the images along this axis, it is possible to construct the three-dimensional image shown 
in Fig. 3, that is, time-space image I(x, y; t).", Fig. 3). 
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Regarding claim 32, Seki teaches an attribute value is a value that indicates a 
degree of change of an image area in time (TJ0016 lines 3-7: ". . .cutting of the initial 
time-space image is performed in all directions. ..the cutting plane is made of a helix 
plane along the movement trace of the object, and this plane completely contains the 
movement vector of the object, and it contains all of the information about the movement 
trace"). 

Regarding claim 34, Seki teaches the attribute value is a pixel value (](0012 lines 
1-2: ". . .a portion of the image of moving object (21) contained in the original 
consecutive images must appear as the trace... only the portion containing angle 9d is 
extracted from each cross-sectional image C(d, t; 6), and a new image is formed", where 
the position of the pixel with the image frame is tracked over a time interval). 

Regarding claim 36, though Seki does not explicitly teach an image input unit, 
one skilled in the art would have understood that an image pickup device, such as a 
camera or camcorder (1(001 1 lines 1-2: ". ..a camcorder is used to take the consecutive 
images that are input to an image processor", contains an image input unit which 
acquires, as the original moving pictures, image shot by a camera and sends the images 
to said image memory). 

Claims 37-39 are rejected under 35 U.S.C. 103(a) as being unpatentable over Seki 
in view of Fels et al. (hereinafter "Fels", "Techniques for Interactive Video Cubism"). 

Regarding claim 37, Seki fails to teach a setting input unit which acquires, via a 
user operation, input of a setting value used to determine the at least one of frames, 
wherein said image conversion unit determines the at least one of frames according to the 
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setting value acquired by said setting input unit. Though Fels does not explicitly teach a 
setting input unit and image conversion unit, one skilled in the art would have understood 
that the input capabilities provided to the user have a corresponding input unit, as well as 
an image conversion unit to process the images displayed (Figs. 1-4), which acquires, via 
user operation, input of a setting value used to determine the at least one of frames, 
wherein said image conversion unit determines the at least one of frames according to the 
setting value acquired by said setting input unit (sec. 1 1 st H lines 1-3: "Using the mouse 
as a virtual trackball, the user is able to rotate or translate the entire scene, the video 
cube, or the cut plane", in which a plane which corresponds with an associated frame, 
Fig. 1, may be selected by user input), where the image conversion unit cuts the box 
space by the surface defined by a function of the setting value acquired by the setting 
input unit (sec. 3.3.1 lines 1-2: "The cut plane allows the user to move a planar window 
inside the video cube and examine the corresponding imagery. . .", where the three- 
dimensional surface is cut by a plane, Fig. 3, in which the processed images are therefore 
displayed, Figs. 1-4, obtained using an image conversion unit), therefore it would have 
been obvious to one of ordinary skill in the art to modify the captured frames generated 
by Seki with the interactive frame interaction provided by Fels because this modification 
would provide the ability to accurately display successive frames of animation or video 
collectively in a three-dimensional format in which the user may interactive with the data 
to define a plane by which to cut the surface of the data to define a representation of the 
change of the images or frames over a time interval thereby enabling temporal analysis of 
the data. 
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Regarding claim 38, Seki teaches a curve that indicates a relation between 
coordinates of points contained in the two-dimensional images and time values thereof 
and a variable of the function is displayed on a screen (]|0012 lines 1-11: "On said 
cross-sectional image C(d, t; 9), a portion of the image of moving object (21) contained 
in the original consecutive images must appear as the trace. ..the object trace on said 
cross-sectional image C(d, t; 9) at a certain time is determined. . . This is called trace 
cross-sectional image L(s, 9; t), and it is found that this image becomes a place 
containing the velocity vector of the moving object. ..this trace cross-sectional image L(s, 
0; t) is an image obtained by cutting said time-space image I(x, y; t) with a helix plane 
along the movement direction of the object", Fig. 5). However, Seki fails to teach a 
setting input unit and a setting value. Though Fels does not explicitly teach a setting input 
unit, one skilled in the art would have understood in view of the teachings of Fels that the 
input capabilities provided to the user have a corresponding input unit, therefore the 
provided input (sec. 1 1 st \ lines 1-3: "Using the mouse as a virtual trackball, the user is 
able to... translate... the video cube, or the cut plane." and sec. 3.3.1 lines 1-2: "The cut 
plane allows the user to move a planar window inside the video cube and examine the 
corresponding imagery. . .", has a corresponding input unit to provide a value that is used 
to define the cut surface, Fig. 4), therefore it would have been obvious to one of ordinary 
skill in the art to modify the captured frames generated by Seki with the interactive frame 
interaction provided by Fels because this modification would provide the ability to 
accurately display successive frames of animation or video collectively in a three- 
dimensional format in which the user may interactive with the data to define a plane by 
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which to cut the surface of the data to define a representation of the change of the images 
or frames over a time interval thereby enabling temporal analysis of the data. 

Regarding claim 39, Seki teaches that based on coordinates of characteristic 
points in the two-dimensional images, the image conversion unit, or image processor, 
cuts the box space by a curve defined by a function of the coordinates of the 
characteristics points (1(0012 lines 1-11: "This is called trace cross-sectional image L(s, 
9; t), and it is found that this image becomes a place containing the velocity vector of the 
moving object. That is, this trace cross-sectional image L(s, 9; t) is an image obtained by 
cutting said time-space image I(x, y; t) with a helix plane along the movement direction 
of the object ", Fig. 5). However, Seki fails to teach a setting input unit and a setting 
value. Though Fels does not explicitly teach a setting input unit, one of ordinary skill in 
the art would have understood in view of the teachings of Fels that the input capabilities 
provided to the user have a corresponding input unit, therefore the input (sec. 1 1 st \ lines 
1-3: "Using the mouse as a virtual trackball, the user is able to... translate... the video 
cube, or the cut plane.'''' and sec. 3.3. 1 lines 1-2: "The cut plane allows the user to move a 
planar window inside the video cube and examine the corresponding imagery. . .", has a 
corresponding input unit to provide a value that is used to define a certain portion of the 
cut surface, Fig. 4), therefore it would have been obvious to one of ordinary skill in the 
art to modify the captured frames generated by Seki with the interactive frame interaction 
provided by Fels because this modification would provide the ability to accurately 
display successive frames of animation or video collectively in a three-dimensional 
format in which the user may interactive with the data to define a plane by which to cut 
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the surface of the data to define a representation of the change of the images or frames 
over a time interval thereby enabling temporal analysis of the data. 



Response to Arguments 

Applicant's arguments filed 2/14/08 have been fully considered but they are not 
persuasive. 

The 35 U.S. C. 101 rejection of claims 40 and 41 has been withdrawn due to the 
amendment to the claims which provides a computer-readable medium encoded with a 
computer program. 

The applicant argues on pg. 11 2 nd ^ lines 5-8 - 3 rd line 1 of the remarks that the 
amendments to claims 1, 17, 40 and 41 to overcome the 35 U.S.C. 1 12 second paragraph 
rejection provided in the previous Office Action, and are supported by the Specification. 
However, the claim language added in the amendment, as provided in claim 1 : 
"projecting a first image that appears on the cut surface onto a first plane perpendicular to 
the time axis; varying the cut surface in time; projecting a second image that appears on 
the varied cut surface onto a second plane perpendicular to the time axis; and outputting 
the first and second images appearing on the first and second planes as new moving 
pictures.", is not supported by the Specification as suggested by the applicant's remarks 
in 3 rd Tl lines 6-15, therefore claims have been rejected under 35 U.S.C. 1 12 first 
paragraph for calming subject that was not described in the originally filed specification. 

The applicant argues on pg. 12 2 nd \ lines 4-5 of the remarks that Seki's plane (1) 
is not perpendicular to the time axis t. However, applicant's arguments are persuasive, 
and claims 1-3 and 40 were indicated as allowable. 
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The applicant argues on pg. 13 2 nd If line 1 - 3 rd ^ line 8 of the remarks that Seki 
describes that image (L) is intended to represent a locus of movement of an object and as 
such, cannot be perpendicular to the time axis t, and would be misconstruing the 
reference in a manner contradictory with its very purpose because projecting Seki's cross 
section image (C) into a plane perpendicular to the time axis would defeat the stated 
purpose of Seki: that is to extract an object movement trace, in which if the cross section 
image (C) were projected into a plane perpendicular to the time axis as the Office Action 
asserts, there would be no information about the speed and direction of the object and 
Seki's device would not work for its intended purpose. However, applicant's arguments 
are persuasive, and claims 1-3 and 40 were indicated as allowable. 

The applicant argues on pg. 14 1 st ^ line 8 - 2 nd ]| line 9 of the remarks that Seki 
does not teach or suggest how single images are made into moving images. However, 
Seki teaches that for each of a plurality of moving objects, a trace cross-sectional image 
is generated, therefore a plurality of moving pictures are created for the plurality of 
objects rendered in the scene. 

The applicant argues on pg. 15 1 st U lines 6-7 of the remarks that Seki does not 
teach a surface that contains a plurality of points each of which differs from the other in 
time value. However, Seki illustrates in Fig. 4 that data is captured from images (11, 12, 
13), which are clearly shown in Fig. 4 to be oriented along a time axis, in which the cut 
surface thereby contains points 21 form each respective instance in time. 
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Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to SAID BROOME whose telephone number is (571)272- 
293 1 . The examiner can normally be reached on M-F 8:30am-5pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
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